A Human-is-the-Loop Approach for Semi-Automated Content Moderation

Online social media has been recognized as a valuable information source for disaster management whose volume, velocity and variety exceed manual processing capacity. Current machine learning systems that support the processing of such data generally follow a human-in-the-loop approach, which has several inherent limitations. This work applies the human-is-the-loop concept from visual analytics to semi-automate a manual content moderation workflow, wherein human moderators take the dominant role. The workflow is instantiated with a supervised machine learning system that supports moderators with suggestions regarding the relevance and categorization of content. The instantiated workflow has been evaluated using in-depth interviews with practitioners and serious games. which suggest that it offers good compatibility with work practices in humanitarian assessment as well as improved moderation quality and higher flexibility than common approaches.